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ABSTRACT 


This  study  investigated  the  extent  to  which  the  Instruc¬ 
tional  Systems  Development  (ISD)  process  for  task  selection 
is  being  used  to  determine  tasks  that  are  trained  at  Radioman 
Class  A  School.  Data  Correlation  Sheets  containing  the 
criteria  ratings  by  the  subject  matter  experts  for  all  general 
radioman  tasks  formed  the  data  base.  The  data  were  evaluated 
using  discriminant  analysis,  correlation  analysis,  and  factor 
analysis  contained  in  the  Statistical  Package  for  the  Social 
Sciences.  The  results  indicated  that  the  number  of  selection 
criteria  can  be  greatly  reduced  and  still  yield  proper  classi¬ 
fication  of  the  tasks.  Although  the  ISD  process  is  doing  an. 
adequate  job  of  task  selection  for  the  school,  this  research 
discovered  that  a  proper  combination  of  fewer  criteria  may 
improve  the  efficiency  and  effectiveness  of  the  process. 
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INTRODUCTION 


Importance  of  Entry-Level  Training  for  Radiomen 

The  focus  of  this  thesis  was  directed  specifically 
toward  radioman  training  due  to  the  unique  character  and 
criticality  of  the  rating.  Because  of  the  continuous  advance¬ 
ments  and  improvements  in  the  telecommunications  industry, 
the  training  of  the  operators  must  be  constantly  scrutinized 
and  evaluated  to  keep  abreast  with  technology. 

Over  the  last  decade  the  Navy  has  witnessed  its  tele¬ 
communications  expand  from  use  of  only  the  high  frequency 
portion  of  the  spectrum  for  long  haul  communications  to 
its  current  use  of  the  ultra  and  super  high  frequency  ranges 
for  the  same  type  of  communications  through  the  employment 
of  satellite  communications  systems,  encryption  devices, 
decryption  devices,  and  other  equipments  all  designed  to 
produce  more  rapid  and  efficient  communications. 

To  operate  and  maintain  the  complex  array  of  tele¬ 
communications  equipment  in  the  Navy's  inventory,  the  human 
factor  of  the  equation  must  not  be  ignored.  Training  of 
radiomen,  especially  at  the  entry-level  phase,  should  pre¬ 
pare  the  student  to  meet  the  challenge  with  appropriate 
understanding  and  the  necessary  level  of  skills  to  accomplish 
tasks  expected  of  his  or  her  skill  level.  Poorly  or  in¬ 
adequately  trained  radiomen  sent  to  their  respective  assign¬ 
ments  would  only  lead  to  an  inefficiency  within  the  overall 
system.  It  is  imperative,  therefore,  that  radiomen  receive 


< 
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the  proper  amounts  of  training  prior  to  reporting  for  duty 
and  that  the  tasks  selected  for  that  training  be  appropriate 
for  the  job. 

Early  in  a  radioman's  career,  assignments  may  be  expected 
to  naval  telecommunications  centers  or  naval  communications 
stations  ashore  or  to  any  of  various  ship  types  at  sea.  On 
the  average,  of  the  students  graduating  from  Radioman  Class 
A  School,  40%  are  assigned  to  shore  installations  while  the 
remaining  60%  are  assigned  to  sea  duty.^  Duties  ashore  are 
primarily  concerned  with  sending,  receiving,  and  processing 
message  traffic,  whereas  duties  at  sea  range  from  the  set  up 
and  operation  of  radioteletype  circuits  with  appropriate 
cryptographic  equipment,  to  publication  corrections  and 
message  distribution  (Radioman  3  &  2,  1978).  In  times  of 
emergency  or  increases  of  threat  in  the  environment,  the 
radioman  must  be  able  to  act  quickly  and  effectively  in 
establishing  and  maintaining  point-to-point  communications. 

Doctrine  for  Selecting  Tasks  for  Training 

Some  tasks  associated  with  a  rating  are  rarely  per¬ 
formed  and  if  neglected  would  cause  little  or  no  job  degrada¬ 
tion,  while  the  criticality  of  adequate  performance  on  other 
tasks  makes  training  for  them  essential.  Budgetary  and  time 
constraints,  however,  impose  certain  restrictions  which  make 


^Indicated  during  personal  communications  with  RMC  Hendricks 
of  Radioman  Class  A  School  in  December,  1979. 
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necessary  the  decision  as  to  which  tasks  get  chosen  for 
training  and  which  do  not.  Thus,  a  selection  process  must 
be  utilized.  The  purpose  of  this  selection  process  should 
be  to  ensure  that  the  important  tasks  will  receive  instruc¬ 
tion  and  to  avoid  wasting  instructional  resources  on  unimpor¬ 
tant  tasks  (CNET,  1978). 

Selection  of  training  tasks  in  the  Navy  is  done  by  a 
process  called  Instructional  Systems  Development  (ISD).  This 
is  a  systematic  set  of  procedures  for  completely  designing 
instructional  programs.  The  full  process  includes  five 
major  phases:  analysis,  design,  development,  implementation, 
and  control.  The  selection  step  falls  under  the  analysis 
phase  and  contains  specific  tools  and  criteria  for  making 
selection  decisions.  In  addition,  the  ISD  process  also  deter¬ 
mines  the  appropriate  instructional  setting  for  those  tasks 
selected  for  training.  Class  A  School  is  delegated  those 
tasks  appropriate  to  vestibule  training.  Class  C  School  is 
reserved  for  tasks  in  which  advanced  level  training  is  more 
appropriate,  and  on-the-job  training  is  designated  for  those 
tasks  for  which  no  formal  schooling  is  deemed  necessary. 

Research  Objectives 

Radioman  Class  A  School  has  recently  come  under  criticism 
for  alledgedly  producing  graduates  who  do  not  possess  the 
basic  knowledge  and  skills  needed  to  operate  standard  communi¬ 
cations  systems  or  function  adequately  in  the  communications 
environment  (COMNAVAIRPAC,  1979).  This  leads  to  an  important 
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question:  How  influential  are  the  task  selection  criteria 
for  selection  of  tasks  for  vestibule  training  in  the  case 
of  Radioman  Class  A  School? 

To  answer  the  question  of  task  selection  effectiveness, 
specific  research  objectives  were  created: 

1.  To  what  extent  do  prescribed  ISD  procedures  for 
task  selection  actually  determine  the  tasks  that  are  trained? 

2.  How  do  decision-makers  actually  use  the  ISD  selec¬ 
tion  criteria  for  decision  making? 

3.  Can  the  ISD  decision  criteria  be  used  more  effectively 
for  selecting  tasks  for  training? 
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ISD  PROCEDURES  FOR  TASK  SELECTION 

Task  Description 

For  a  greater  appreciation  of  task  evaluation  and 
selection,  the  reader  should  be  familiar  with  the  official 
criteria  for  the  identification  of  job  tasks  (CNET,  1976): 

Task  Definition 

A  task  is  the  lowest  level  of  behavior  in  a  job  that 
describes  the  performance  of  a  meaningful  function  of  the 
job. 

Task  Characteristics 

1.  A  task  statement  is  a  statement  of  a  highly 
specific  action. 

2.  A  task  has  a  definite  beginning  and  end. 

3.  Tasks  are  performed  in  relatively  short  periods 
of  time. 

4.  Tasks  must  be  observable. 

5.  A  task  must  be  measurable.' 

6.  Each  task  is  independent  of  other  tasks  (performed 
for  its  own  sake). 

Selection  Criteria 

Prior  to  the  actual  task  selection  process,  a  job  task 
inventory  (JTI)  is  composed  through  studies  and  surveys  in 
the  field.  For  radioman  training,  the  JTI  identifies  general 
radioman  tasks,  that  is  tasks  which  do  not  require  any  special 
NECs.  Once  these  tasks  are  compiled,  the  lists  are  submitted 
to  groups  of  Subject  Matter  Experts  (SME)  for  rating  and 
evaluation. 
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The  ratings  are  recorded  on  Data  Correlation  Sheets. 

Each  sheet  has  several  columns  for  the  criteria  which  are 
rated  on  a  scale  from  1  to  5.  Generally  speaking,  the  higher 
the  rating  (i.e.,  the  closer  to  5)  the  higher  the  priority 
for  training.  The  following  criteria  are  utilized  as  an  aid 
in  selecting  tasks  for  training  and  are  listed  with  their 
respective  codes: 

Probable  Consequence  of  Inadequate  Performance.  This 
is  a  measure  of  the  effect  a  job  may  have  on  the  mission 
if  it  is  unsatisfactorily  performed. 

1.  Negligible 

2.  Rather  trivial 

3.  Fairly  serious 

4.  Very  serious 

5.  Disastrous 

Delay  Tolerance.  This  is  a  measure  of  how  much  delay 
may  be  tolerated  between  the  time  the  need  for  task  performance 
arises  and  the  time  actual  performance  must  begin. 

1.  Extremely  high  (Performance  may  be  delayed  for  a 
considerable  period) 

2.  High 

3.  Average 

4.  Low 

5.  Extremely  Low  (Task  requires  immediate  performance) 
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Learning  Difficulty.  This  indicates  the  effort,  time 
and  assistance  needed  by  a  student  to  acquire  proficiency  in 
the  performance  of  the  task. 

1.  Extremely  low  (No  training  is  required) 

2 .  Low 

3.  Average 

4.  High 

5.  Extremely  high  (Training  is  essential) 

Probability  of  Deficient  Performance.  This  is  the 

likelihood  that  the  job  performer  will  perform  the  task 
unsatisfactorily . 

1.  Rarely,  if  ever. 

2.  Less  often  than  other  tasks. 

3.  About  as  often  as  other  tasks. 

4.  More  often  than  other  tasks. 

5.  Very  often. 

Time  Between  Job  Entry  and  Task  Performance.  This  is 
the  lapse  of  time  that  occurs  between  the  time  the  person 
reports  for  duty  in  a  job  and  the  time  he  or  she  actually 
carries  out  the  particular  task, 

1.  Task  not  yet  performed. 

2.  Task  first  performed  beyond  four  years  after 
assignment. 

3.  Task  first  performed  between  two  and  four  years 
after  assignment. 

4.  Task  first  performed  between  one  and  two  years 
after  assignment. 
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5.  Task  first  performed  during  first  year  after 
assignment . 

Decay  Rate.  This  is  an  estimate  of  the  length  of 
time  a  student  is  able  to  retain  knowledge  and  skills  learned. 

1.  Extremely  high  (Knowledge  is  rapidly  lost) 

2.  High 

3.  Average 

4 .  Low 

5.  Extremely  low  (Knowledge  is  retained  for  a  long 
period) 

Complexity .  This  is  a  measure  of  how  complicated 
the  task  is  to  perform. 

1.  Extremely  low  (Simple) 

2 .  Low 

3.  Average 

4 .  High 

5.  Extremely  high  (Complex) 

Frequency  of  Performance.  This  is  a  measure  of  the 
expected  rate  at  which  the  task  is  likely  to  be  performed. 

1.  Never  performed, 

2.  Performed  less  than  once  per  month. 

3.  Performed  at  least  once  per  month  but  less  than 
twice  per  week. 

4.  Performed  twice  per  week. 

5.  Performed  more  than  twice  per  week.  (Daily) 
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Subject  Matter  Expert  Evaluation  Period 


Prior  to  selection  of  the  tasks  included  in  the  current 

course  at  Radioman  Class  A  School,  Subject  Matter  Experts 

from  the  school  evaluated  all  the  tasks  of  the  JTI  on  the 

Data  Correlation  Sheets.  The  evaluation  occurred  over  a 

three-month  period  from  August  1976  to  1  November  1976. 

Once  each  week  the  SMEs  gathered  and,  as  a  group,  rated 

tasks  on  each  of  the  criteria.  Of  the  total  group  of  17 

2 

SMEs,  only  five  at  any  time  would  rate  any  given  task. 

After  rating  the  tasks  on  the  criteria,  the  SMEs  were 
asked  to  recommend  the  appropriate  training  facility:  "A" 
School,  "C"  School,  Fleet  Training  or  Functional  School, 
formal  on-the-job-training,  or  no  formal  training.  The 
SMEs  did  not  actually  select  tasks  for  training,  they  only 
recommended  the  instructional  setting  (Kennedy,  Kalivoda, 
Dickie,  Drummer,  and  Duember,  1978). 

Selection  Judgment 

The  ratings  assigned  to  the  task  criteria  were  based 
upon  the  SMEs*  experience  and  judgment.  Although  the  SMEs 
did  not  actually  select  the  tasks  to  be  trained,  using  the 
results  of  their  task  criteria  ratings  they  did  make 
recommendations  regarding  such  task  selection. 


o 

As  indicated  in  personal  communications  with  Mr.  Richard 
Lund,  the  Front  End  Analysis  Project  Manager  at  the  time, 
22  February  1980. 
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The  selection  of  tasks  for  training  is  a  judgment  which 
requires : 

— A  clear  understanding  of  the  duties,  tasks,  and 
elements  that  make  up  the  job. 

— Analysis  of  collected  data  by  individuals  who  are 
familiar  with  the  job. 

— An  understanding  of  the  resources  and  the  respon¬ 
sibility  of  the  command  that  makes  the  training  choice. 

— A  clear  understanding  of  when  and  under  what  condi¬ 
tions  training  is  appropriate.  CCNET,  1978,  p.  11). 

The  Data  Correlat ion  Sheet 

The  Data  Correlation  Sheets  derive  their  tasks  from 
those  listed  in  the  associated  JTI.  The  Data  Correlation 
Sheet  itself  breaks  down  into  a  number  of  columns  for  each 
task  listed  thereon.  See  Figure  1. 

The  first  column  indicated  the  task  number  assigned 
to  the  specific  task  in  the  JTI.  The  letter  in  this  alpha¬ 
numeric  group  signifies  that  the  task  is  a  member  of  one 
of  the  following  groups: 

A  -  Administrative  Tasks 
B  -  Shore  Tasks 
C  -  Sea  Tasks 

D  -  Traffic  Handling  -  Circuit  Operations  Tasks 
£  -  Automated  Systems  Tasks 


3ATA  CORRELATION  SHEET 


Figure  1.  Example  of  a  Data  Correlation  Sheet 


The  "task"  column  contains  the  title  of  the  task.  The 
Navy  Occupational  Task  Analysis  Program  (NOTAP)  number  follows 
and  contains  no  significance  for  this  thesis. 

The  next  three  columns  are  composed  of  the  percentages 
within  rate  groups  who  perform  the  given  task  as  determined 
by  the  NOTAP  surveys . 

Following  these  percentage  columns  is  the  ’’Existing 
Skill  Level"  column  which  indicates  the  minimum  rate  required 
to  perform  the  task.  The  code  ranges  from  2  to  9  and  stands 
for  E-2  to  E-9 . 

The  task  criteria  ratings  described  earlier  constitute 
the  entries  for  the  next  seven  columns. 

The  "Suggested  Skill  Level"  consists  of  the  recommended 
rate  as  judged  by  the  SMEs.  In  most  cases,  the  figures  in 
this  column  were  found  to  be  identical  to  those  in  the 
"Existing  Skill  Levels"  column. 

The  final  two  columns,  "Recommended  Instructional 
Setting"  and  "Remarks"  were  left  blank  on  all  sheets. 
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METHOD 


Approach 

To  analyze  the  task  selection  process,  the  actual  out¬ 
put  of  the  Radioman  Class  A  School  instructional  systems 
development  process  in  the  form  of  finished  Data  Correlation 
Sheets  was  used.  The  ratings  on  the  selection  criteria  were 
analyzed  to  determine  the  degree  to  which  they  predicted  the 
training  decision  for  each  task. 

Of  the  nearly  600  individual  tasks  evaluated  for  all 
radioman  skill  levels,  this  study  considered  125.  Using  the 
Job  Task  Inventory  (JTI)  as  a  reference  indicating  which 
tasks  were  currently  being  taught  in  the  Class  A  School 
and  the  Data  Correlation  Sheets  as  a  guide  to  their  desig¬ 
nated  skill  levels  the  author  was  able  to  restrict  the  field 
of  tasks  to  be  investigated  (CO,  NETSCPAC,  1979).  Among 
those  tasks  that  were  examined  were  all  the  tasks  specified 
as  E-2  skill-level  tasks,  those  currently  taught  at  the 
Radioman  Class  A  School  as  well  as  those  not  taught.  Addi¬ 
tionally,  those  tasks  assigned  a  higher  existing  skill  level 
than  E-2  and  which  were  also  taught  at  the  school  were  also 
included  in  the  analysis.  Tasks  which  required  a  minimum 
skill  level  above  E-2  and  weren't  taught  at  the  school  were 
not  used,  under  the  assumption  that  such  skills  were  irrel¬ 
evant  to  a  school  primarily  concerned  with  vestibule  training. 

The  dependent  variables  then  consisted  of  three  groups 
or  categories: 
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1.  Tasks  at  the  E-2  skill  level  which  were  not  taught. 

2.  Tasks  at  the  E-2  skill  level  which  were  taught. 

3.  Tasks  above  the  E-2  skill  level  which  were  taught. 
The  independent  variables  manipulated  were  the  task 

criteria  rated  by  the  SMEs  as  well  as  the  percentage  of 
E-2s  to  E-4s  in  the  Radioman  rating  that  perform  the  task 
as  taken  from  the  associated  NOTAP  survey.  The  independent 
variables  used  were: 

1.  Percentage  of  E-2  to  E-4  Performing  Task 

2.  Probable  Consequence  of  Inadequate  Performance 

3.  Delay  Tolerance 

4.  Learning  Difficulty 

5.  Probability  of  Deficient  Performance 

6.  Time  Between  Job  Entry  and  Task  Performance 

7.  Decay  Rate 

8.  Complexity 

9.  Frequency  of  Performance 


Analytic  Procedure 


Various  statistical  methods  available  in  the 


Statistical  Package  for  the  Social  Sciences  (SPSS)  were  used 
for  analysis.  The  programs  and  data  were  run  using  batch 
processing  on  the  IBM  360  series  computer  located  at  the 
Naval  Postgraduate  School.  The  source  of  the  tables  in  the 
next  chapter  is  the  computer  output  from  these  programs. 

Discriminant  analysis  was  performed  to  see  if  the  tasks 
belonging  to  one  of  the  three  dependent  variable  groups  could 
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be  statistically  distinguished  from  those  belonging  to  the 
others  and  which  variables  influenced  this  discrimination 
the  most.  This  analysis  was  also  done  to  determine  how  well 
tasks  could  be  classified  correctly  as  to  their  group  member¬ 
ships  . 

Correlation  analysis  was  conducted  for  the  purposes  of 
noting  significant  relationships  among  the  variables. 

Factor  analysis  was  done  to  try  to  consolidate  the  in¬ 
dependent  variables  into  a  smaller  list  of  criteria  which 
form  distinct  attributes. 

All  tasks,  with  their  factor  scores  from  the  factor 
analysis,  were  entered  back  into  a  discriminant  analysis. 

The  results  of  this  analysis  were  then  compared  with  the 
original  discriminant  analysis. 

Description  of  the  Analytic  Techniques  Used 

Discriminant  Analysis 

The  mathematical  objective  of  discriminant  analysis  is 
to  weight  and  linearly  combine  the  discriminating  variables 
in  some  manner  so  that  the  groups  are  forced  to  be  as 
statistically  distinct  as  possible.  Discriminant  analysis 
provides  a  classification  aspect  whereby  one  may  insert  a 
case  with  unknown  membership  and  receive  an  assignment  for 
that  case  into  a  group  based  upon  a  derived  set  of  classi¬ 
fication  functions.  It  also  uses  several  statistical  tests 
for  measuring  the  success  with  which  the  discriminating 
variables  actually  discriminate  when  combined  into  the 
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discriminant  functions.  Variables  which  contribute  the 
most  to  differentiation  along  the  respective  functions  are 
identified  (Klecka,  1975). 

Correlation  Analysis 

It  is  often  desirable  to  observe  and  measure  the  assoc¬ 
iation  which  occurs  between  two  or  more  statistical  series. 

The  association  between  series  may  be  established  and 
measured  by  means  of  regression  analysis.  Pearson  correlation 
is  one  of  the  variations  of  regression  analysis  used  to  dis¬ 
cern  variable  relationships.  It  yields  the  correlation  co¬ 
efficient  value  of  variable  pairs  as  well  as  their  statistical 
significance  (Arkin  &  Colton,  1960). 

Factor  Analysis 

Factor  analysis  is  employed  to  combine  many  variables 
into  smaller  groups  from  which  to  make  statistical  inferences 
or  to  apply  further  analytical  techniques.  Factor  analysis 
may  be  useful  in  any  of  a  number  of  ways.  It  was  applied 
to  this  thesis  in  the  following  ways: 

1.  Data  Reduction  —  Consolidation  of  many  variables 
into  a  few,  easily  handled  variables. 

2.  Data  Transformation  —  Translation  of  results  into 
factor  scores  to  be  further  processed  through 
discriminant  analysis. 

3.  Exploratory  Uses  —  To  uncover  unsuspected  relation¬ 
ships  which  may  seem  startling  at  first  glance,  but 
are  actually  common  sense  (Rummel,  1970). 
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RESULTS 


Descriptive  Statistics 

To  present  an  overview  of  the  individual  selection 
criterion  ratings,  Table  1  presents  the  means  and  standard 
deviations  of  each  task  selection  criterion  by  criterion 
group  and  also  as  a  total  figure  for  all  the  categories  com¬ 
bined.  The  "Percentage  of  E-2s  to  E-4s  Performing"  variable 
used  a  scale  ranging  from  1  to  100,  while  the  rest  employed 
scales  from  1  to  5. 

Group  1  is  comprised  of  all  those  tasks  requiring  a 
minimum  skill  level  of  E-2  for  proper  performance  which 
were  not  being  taught  in  the  course  at  Radioman  Class  A 
School.  Classification  of  this  group  and  the  others  was 
based  upon  the  1979  Job  Task  Inventory  for  general  radioman 
tasks  and  the  Data  Correlation  Sheets.  Group  2  consists 
of  the  remaining  E-2  tasks,  all  of  which  were  being  taught 
in  the  school.  Group  3  contains  all  the  other  tasks  that 
were  being  taught  and  which  were  determined  to  have  a  mini¬ 
mum  existing  skill  level  higher  than  E-2. 

Prediction  and  Policy  Capturing 

Discriminant  analysis  was  performed  using  the  Wilks' 
stepwise  method.  The  use  of  a  stepwise  procedure  produces 
an  optimal  set  of  variables  being  selected.  It  is  an  efficient 
means  of  approximately  locating  the  best  set  of  discriminating 
variables.  The  Wilks'  lambda  is  a  measure  of  group  discrim¬ 
ination  (Nie,  Hull,  Jenkins,  Steinbrenner ,  &  Brent,  1975). 
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Table  1 


Means  and  Standard  Deviations  of  Criterion  Groups 
on  the  Task  Selection  Criteria 


Task  Criterion 

Group  1 

Group  2 

Group  3 

Total 

Percentage  of  E-2  to 

E-4  Performing 

36.93 

28.77 

38.09 

29.54 

40.26 

36.45 

38.30 

31.15 

Probable  Consequence  of 
Inadequate  Performance 

2.56 

1.01 

2.26 

1.01 

3.17 

1.22 

2.62 

1 . 13 

Delay 

Tolerance 

2.88 

.96 

2.96 

1.14 

2.77 

.91 

2.88 

1.01 

Learning 

Difficulty 

2.35 

.87 

2.49 

.80 

2.63 

.91 

2.48 

.86 

Probability  of  Deficient 
Performance 

2.63 

.82 

2.62 

.99 

2.60 

.85 

2.62 

.89 

Time  Between  Job  Entry 
and  Task  Performance 

4.93 

.26 

4.96 

.20 

3.94 

1.00 

4.66 

.72 

Decay 

Rate 

1.33 

.61 

1.43 

.80 

1.40 

.50 

1.38 

.66 

Complexity 

1.51 

.86 

1.66 

.94 

2.00 

.97 

1.70 

.93 

Frequency  of 

Performance 

4.42 

.85 

4.91 

.28 

4.66 

.48 

4.67 

.62 

Note  1 :  In  each  pair  of  figures  the  mean  is  on  top  and  the 

standard  deviation  is  below. 

Note  2:  The  groups  are  coded  as  follows: 

Group  1  —  Tasks  at  the  E-2  skill  level  which  were 
not  taught. 

Group  2  —  Tasks  at  the  E-2  skill  level  which  were 
taught . 

Group  3  —  Tasks  above  the  E-2  skill  level  which 
were  taught. 
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Two  functions  were  derived  from  the  discriminant 
analysis.  Table  2a  displays  the  standardized  discriminant 
function  coefficients.  Ignoring  the  sign,  each  coefficient 
indicates  the  relative  contribution  of  its  associated  variable 
to  the  particular  function  since  they  are  measured  on  the 
same  scales.  The  sign  just  signifies  whether  the  variable 
makes  a  positive  or  negative  contribution.  The  coefficients 
capture  the  policy  used  by  the  ISD  personnel  in  weighting 
the  task  selection  criterion  variables. 

The  next  table,  Table  2b,  provides  additional  information 
for  evaluating  the  effectiveness  of  the  two  derived  functions. 
The  eigenvalue  is  a  measure  of  the  relative  importance  of  the 
function.  The  sum  of  the  eigenvalues  is  a  measure  of  the 
total  variance  existing  in  the  discriminating  variables.  The 
percentage  of  variance  yields  an  easy  reference  to  the 
relative  importance  of  the  associated  function.  The  canonical 
correlation  is  another  measure  of  the  function's  ability  to 
discriminate  among  the  groups.  Wilks  lambda  is  an  inverse 
measure  of  the  discriminating  power  in  the  original  variables 
which  has  not  yet  been  removed  by  the  discriminant  functions  — 
the  larger  lambda  is  the  less  information  remains.  The 
significance  of  the  chi-square  values  indicates  that  the 
functions  predict  the  dependent  variable  considerably  better 
than  chance  (Nie,  et  al.,  1975). 

Classification  of  the  cases  applying  the  discriminant 
functions  is  depicted  in  Table  2c.  This  table  shows  the 
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Table  2 

Discriminant  Analysis  of  the  Task  Selection  Criteria 


a.  Standardized  Canonical  Discriminant  Function  Coefficients 


Independent  Variable 

Function  1 

Function 

2 

Probable  Consequence  of 
Inadequate. Performance 

.  301 

-.337 

Delay 

Tolerance 

-.257 

-.010 

Probability  of  Deficient 

Performance 

-.316 

-.058 

Time  Between  Job  Entry 

and  Task  Performance 

-.891 

-.014 

Complexity 

.312 

.442 

Frequency  of 

Performance 

.248 

.949 

b.  Statistics  Related  to 

the  Discriminant 

Functions 

Statistic 

Function  1 

Function 

2 

Eigenvalue 

.812 

.  173 

Percent  of  Variance 

82.45 

17.55 

Cumulative  Percent 

82.45 

100.00 

Canonical  Correlation 

.67 

.38 

After  Function 

0  After  Function 

Wilks'  Lambda 

.47 

.85 

Chi-Square 

90.11 

19.06 

Degrees  of  Freedom 

12 

5 

Significance 

.000 

.002 
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c.  Discriminant  Classification  of  Tasks  Versus  Actual  Task 
Group  Membership 


Actual  Group 

#  of  Cases 

Predicted 

Group 

Membership 

Group  1 

Group 

2  Group  3 

Group  1 

43 

19 

24 

0 

44.2% 

55.8% 

0.0% 

Group  2 

47 

11 

35 

1 

23.4% 

74.5% 

2.1% 

Group  3 

35 

2 

8 

25 

5.7% 

22.9% 

71.4% 

Note  1.  For  a  breakdown  of  Groups  1,  2,  &  3,  see  Table  1. 

Note  2.  Percent  of  "grouped"  cases  correctly  classified:  63.2%. 
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number  of  cases  (tasks)  which  belong  to  each  group  and  the 
number  and  the  percentage  of  cases  correctly  and  incorrectly 
placed.  The  overall  percentage  of  correct  placements  was 
63.2,  whereas  the  expected  (random)  placement  was  33.3 
percent.  Thus,  the  calculated  placement  is  200  percent 
better  than  chance. 

Factor  Analysis  and  Factor  Scoring  of  Task  Selection  Criteria 

Factor  analysis  was  used  to  determine  what  underlying 
factors  or  principles  were  used  in  determining  which  tasks 
would  be  taught.  The  purpose  of  the  analysis  was  to  develop 
a  set  of  fewer  selection  criteria  that,  being  more  distinct 
(less  overlap),  might  prove  to  be  more  effective  and/or  more 
efficient  in  the  classification  of  job  tasks  for  training. 

Table  3  presents  the  Pearson  correlation  matrix  showing 
the  first-order  relationship  among  the  variables  that  form 
the  basis  for  the  factor  analysis.  It  is  conventional  to 
regard  a  probability  of  0.05  as  the  critical  level  of  sig¬ 
nificance  (Tippett,  1952).  The  significance  was  calculated 
for  each  criterion  pair  on  the  basis  of  all  125  cases  except 
for  those  pairs  involving  the  "Percentage  of  E-2s  to  E-4s 
Performing"  due  to  missing  values  for  that  variable  in  20 
of  the  cases. 

The  method  of  principal  factoring  (without  interation 
and  with  varimax  rotation)  was  used  to  extract  the  factors. 
Table  4a  presents  the  weights  and  variance  accounted  for 
by  the  nine  factors  calculated.  Clearly  the  first  three 
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Table  3 

Correlation  Matrix  of  the  Independent  Variables 
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Table  4 


Factor  Analysis 


a.  Factor  Weights 


Factor 

Eigenvalue 

%  of  variance 

Cumulative  % 

1 

3.00 

33.4 

33.4 

2 

1.53 

17.0 

50.4 

3 

1.26 

14.0 

64.4 

4 

.93 

10.3 

74.7 

5 

.74 

8.3 

83.0 

6 

.55 

6.2 

89.2 

7 

.48 

5.3 

94.5 

8 

.28 

3.1 

97.6 

9 

.21 

2.4 

100.0 

Varimax  Rotated  Factor  Matrix 


Independent  Difficulty  Immediacy  Consequences 

variablesa  (Factor  1)  (Factor  2)  (Factor  3) 


E2E4 

-.069 

INADPERF 

.135 

DELAYTOL 

.262 

LEARNDIF 

.786 

DEFPERF 

.696 

TIME 

-.079 

DECAYRT 

.900 

CMPLXTY 

.869 

FREQ 

-.129 

.778 

-.002 

.289 

.794 

.793 

.094 

.186 

.132 

.081 

.171 

.164 

-.783 

-.083 

-.073 

-.097 

.172 

.389 

-.353 

aFor  full  variable  names  see  Table  3,  note  1. 
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factors  possess  the  most  influence,  comprising  nearly  two- 
thirds  of  the  total  variance.  The  program  retained  and 
printed  only  components  with  eigenvalues  greater  than  or 
equal  to  1.0,  hence,  only  the  first  three  factors  were  con¬ 
sidered  by  the  program  for  the  remainder  of  its  analysis 
(Kim,  1975). 

Table  4b  portrays  the  coefficients  of  each  criterion 
as  they  lie  in  each  of  the  three  factors.  For  Factor  1, 
the  criteria  of  decay  rate,  complexity,  learning  difficulty, 
and  probability  of  deficient  performance  carried  the  most 
weight  and,  in  combination,  seemed  to  indicate  a  difficulty 
factor.  Hence,  Factor  1  was  entitled  "Difficulty."  Factor 
2  is  dominated  by  delay  tolerance  and  the  percentage  of  E-2s 
to  E-4s  performing  the  task.  This  seemed  to  indicate  a 
feeling  of  immediacy.  Thus,  "Immediacy"  was  the  name  given 
to  that  factor.  Probable  consequence  of  inadequate  perfor¬ 
mance  and  time  between  job  entry  and  task  performance  were 
the  strongest  coefficients  in  Factor  3.  "Consequences"  was 
the  word  deemed  most  descriptive  of  Factor  3.  Varimax 
rotation  has  the  effect  of  clustering  variables.  The 
overall  outcome  is  a  reduction  in  the  number  of  inputs  to 
be  analyzed  further. 

The  cases  were  individually  factor  scored  using  the 
three  factors.  Then,  the  factors  were  injected  into  a 
discriminant  analysis  using  the  factor  scores  for  each 
task. 
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Table  5a  shows  that  the  immediacy  factor  and  the  con¬ 
sequences  factor  were  the  most  discriminating  of  the  three. 
Function  coefficients  were  assigned  accordingly. 

Table  5b  indicates  that  Function  1  is  overwhelmingly 
superior  in  discrimination  to  Function  2  through  the  extreme 
difference  between  eigenvalues.  Note  that  the  percentage 
of  variance  of  the  first  function  is  nearly  one  hundred 
percent.  There  is  a  correspondingly  large  difference  between 
the  canonical  correlations.  Notice  also  that  there  is  an 
extremely  high  level  of  significance  for  the  first  function, 
but  the  second  function  is  not  significant. 

Table  5c  displays  correctly  and  incorrectly  predicted 
memberships  of  groups  based  on  discrimination  due  to  factor 
analysis.  Note  that  the  overall  percent  of  "grouped"  cases 
classified  correctly  is  1.2%  less  than  that  of  the  task 
criteria  discrimination  shown  in  Table  2c.  However,  from 
the  standpoint  of  efficiency,  the  classification  is  being 
accomplished  with  essentially  one  function  that  has  only 
two  component  variables  (Factors  2  and  3). 

Figures  2  and  3  graphically  depict  the  discrimination 
based  on  task  criteria  and  factor  scores  for  their  respec¬ 
tive  functions.  Notice  how  Function  1  in  Figure  2  clearly 
discriminates  the  tasks  with  existing  skill  levels  higher 
than  E-2  from  all  tasks  on  the  E-2  level.  Function  2  of 
the  same  figure  clearly  segregates  both  groups  of  E-2 
skill  level  tasks  from  each  other.  In  Figure  3,  one  may 


Table  5 


Discriminant  Analysis  of  the  Factor  Analysis  Results 

a.  Standardized  Canonical  Discriminant  Function  Coefficients 
of  the  Factor  Scored  Tasks 


Factor 

Function  1 

Function  2 

Immediacy  (Factor  2) 

-.391 

.929 

Consequences  (Factor  3) 

.971 

.270 

b.  Statistics  Related  to 
Factor  Scored  Tasks 

the  Discriminant 

Functions  of  the 

Statistic 

Function  1 

Function  2 

Eigenvalue 

.532 

.001 

Percent  of  Variance 

99.82 

.18 

Cumulative  Percent 

99.82 

100.00 

Canonical  Correlation 

.59 

.03 

After  Function  0 

After  Function  1 

Wilks  Lambda 

.65 

.99 

Chi-Squared 

51.99 

.12 

Degrees  of  Freedom 

4 

1 

Significance 

.000 

.730 

c.  Discriminant  Classification  of  Factor  Scored  Tasks  Versus 
Actual  Task  Group  Membership 


Actual  Group 

#  of  Cases 

Predicted 

Group 

Membership 

Group  1 

Group 

2 

Group  3 

Group  1 

43 

20 

46.5% 

16 

37.2% 

7 

16.3% 

Group  2 

47 

14 

29.8% 

32 

68.1% 

1 

2.1% 

Group  3 

35 

2 

5.7% 

8 

22.9% 

25 

71.4% 

Note  1.  For  Breakdown  of  Groups  1,  2,  &  3,  see  Table  1. 


Note  2.  Percent  of  "grouped"  cases  correctly  classified: 
61.6%. 
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Scatterplot  of  canonical  discriminant  functions 
of  factor  scored  tasks  evaluated  at  group  means. 


readily  discern  a  negligible  amount  of  effect  among  the 
groups  by  Function  2,  while  Function  1  exhibits  a  fair 
degree  of  separation  among  the  groups. 


DISCUSSION 


Overview 

It  appears  as  though  the  ISD  process  is  successful  in 
the  selection  of  important  tasks  for  training  at  Radioman 
Class  A  School.  The  criteria  of  probable  consequence  of 
inadequate  performance,  delay  tolerance,  time  between  job 
entry  and  task  performance,  complexity,  and  frequency  of 
performance  seem  to  influence  the  selection  decisions  more 
strongly  than  the  others.  However,  some  criteria  used 
in  the  selection  process  detract  from  the  potential  efficiency 
of  the  process.  Their  contribution  is  negligible  in  most 
instances.  There  are  ways  to  increase  efficiency  through 
better  utilization  of  the  more  discriminating  criteria. 

The  results  suggest  that  there  exists  a  clear  discrimi¬ 
nation  between  all  E-2  tasks  and  those  tasks  of  a  higher 
skill  level.  However,  an  unclear  picture  remains  of  the 
discrimination  between  the  E-2  skill  level  tasks  which  are 
selected  for  Radioman  Class  A  School  training  and  those  that 
are  not.  The  decision  for  not  selecting  a  task  for  training 
is  apparently  based  upon  other  criteria  such  as  facilities 
available,  institutional  resources  available  —  both  budgetary 
and  personnel,  etc. 

The  analyses  raised  doubt  about  whether  certain  tasks 
were  rated  higher  following  a  conscious  decision  that  they 
needed  to  be  trained  or  if  the  selection  of  the  tasks  occurred 
because  certain  tasks  had  higher  ratings  from  an  objective 
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rating  process.  One  is  unable  to  ascertain  the  truth  from 
static  data. 

Integration  of  Analytical  Findings 

Examination  of  the  group  means  of  the  criteria  reveals 
that  in  the  majority  of  instances  Group  3  tasks  (the  higher 
skilled,  taught  tasks)  are  easily  distinguishable  from  the 
tasks  in  the  E-2  skill  level  groups  by  a  significant  degree. 
This  is  readily  apparent  in  the  criteria  of  probable  conse¬ 
quence  of  inadequate  performance,  delay  tolerance,  learning 
difficulty,  time  between  job  entry  and  task  performance,  and 
complexity.  By  contrast,  only  a  few  criteria  separate  Group 
2  tasks  from  the  rest,  and  fewer  still  separate  Group  1 
tasks . 

Insight  may  be  gained  from  contrasting  pairs  of  task 
groups  using  the  criteria  means.  Probable  consequences  of 
inadequate  performance  exhibits  a  large  difference  between 
Group  2  and  Group  3  skills.  The  same  holds  for  delay  tolerance 
and  time  between  job  entry  and  task  performance,  although  for 
the  latter,  the  more  important  result  is  that  Group  3  is 
discriminated  from  the  combination  of  Groups  1  and  2  which 
are  nearly  equal  in  value.  Learning  difficulty  displays  its 
greatest  separation  on  Group  3  from  Group  1.  Decay  rate  is 
unique  in  that  it  is  the  only  criterion  which  seems  to  pull 
Group  1  out  from  the  others  with  its  low  mean.  This  makes 
sense  because  it  is  better  to  formally  train  those  tasks 
which  are  more  easily  retained  for  later  use  than  train  the 
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tasks  which  will  only  require  retraining  at  the  job  site  due 
to  knowledge  loss.  Decay  rate  and  frequency  are  the  only 
real  criteria  which,  markedly  contrast  Group  1  with  Group  2. 
Unfortunately,  this  decay  rate  carries  little  importance  in 
either  of  the  discriminant  analyses  performed.  Frequency 
also  differentiates  Groups  1  and  2.  Also,  complexity  shows 
a  consistent  difference  for  each  task  group.  The  probability 
of  deficient  performance  criterion  is  too  evenly  matched 
across  the  board  to  have  any  important  effect  in  the  outcome 
of  the  selection  process. 

For  finer  analysis  of  the  differences  between  the 
groups,  discriminant  and  factor  analysis  were  performed. 

The  discriminant  analysis  of  the  task  criteria  greatly  dis¬ 
criminates  Group  3  from  the  others.  The  high  positive  weight 
assigned  to  probable  consequence  of  inadequate  performance 
in  the  first  function,  when  multiplied  to  the  group  means, 
boosts  the  Group  3  value  considerably  higher  than  the  others. 
Similarly,  the  weight  of  complexity  raises  the  value  of 
Group  3.  The  criterion  of  time  between  job  entry  and  task 
performance  contains  a  large,  negative  weight  which  contributes 
to  a  negative  value  added  to  the  overall  score  for  Group  3, 
but  to  a  much  lesser  extent  than  the  negativity  which  re¬ 
sults  for  both  Groups  1  and  2.  The  second  function  favors 
discrimination  of  Group  2  in  that  the  negative  weight  assigned 
to  the  probable  consequence  of  inadequate  performance  criterion 
and  the  high,  positive  value  associated  with  frequency  of 


performance  contribute  significantly  to  a  higher  overall 
value  for  the  Group  2  scores.  This  second  function  is  con¬ 
siderably  less  important  than  the  first  function.  Thus, 
overall,  this  discriminant  analysis  application  is  best 
suited  for  the  classification  of  Group  3  tasks. 

Checking  the  results  of  the  factor  analysis,  one  sees 
that  the  discrimination  of  groups  is  pretty  much  the  same, 
with  Group  3  being  delineated  the  best.  The  Difficulty  factor 
is  composed,  in  part,  of  learning  difficulty,  decay  rate,  and 
complexity,  all  of  which  have  high  positive  weightings  which 
places  the  value  of  the  Group  3  factor  scores  well  above  the 
other  two.  These  three  criteria  are  the  most  important  in 
the  Difficulty  factor.  In  combination  with  each  other,  it 
would  seem  that  these  criteria  should  be  excellent  in  dis¬ 
criminant  analysis.  The  Immediacy  factor  favors  the  discrimi¬ 
nation  of  Group  2  among  the  others.  The  high  positive 
weighting  of  delay  tolerance  along  with  the  moderately  high 
positive  weighting  of  frequency  of  performance,  boosts  the 
resultant  value  of  Group  2's  factor  scores  higher  than  those 
of  the  other  groups.  Yet,  this  factor,  too,  is  relatively 
insignificant  in  the  discrimination  analysis  that  was  per¬ 
formed.  In  the  Consequences  factor,  the  high  positive  weight 
for  probable  consequence  of  inadequate  performance  together 
with  the  high  negative  weight  for  time  between  job  entry  and 
task  performance  plays  an  inqportant  part  in  yielding  a  high 
value  to  the  Group  3  factor  scores.  The  moderately  high 
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negative  weight  given  to  the  frequency  of  performance  cri¬ 
terion  enables  the  Group  1  factor  scores  to  be  somewhat 
higher  than  Group  2  equations,  but  not  as  high  as  those  for 
Group  3.  A  quick  glance  at  the  Function  1  axis  in  Figure 
3  illustrates  the  point.  The  Consequences  factor  is  the 
single  most  important  discriminating  factor  according  to 
the  discriminant  analysis. 

The  discriminant  analysis  of  the  factor  scores  assigns 
a  high  positive  weight  to  the  Consequences  factor  and  a 
moderately  high  negative  weight  to  the  Immediacy  factor. 

This  has  the  effect  of  maintaining  the  high  amount  of  Group 
3  discrimination  in  the  Consequences  factor  while  something 
interesting  happens  within  the  Immediacy  factor.  Heretofore, 
the  Immediacy  factor  was  a  good  discriminator  of  the  Group 
2  tasks.  With  a  moderately  high  negative  weight  multiplied 
to  the  Immediacy  factor  scores,  the  dominant  criteria  of 
delay  tolerance  and  frequency  of  performance  switch  from 
yielding  a  positive-type  influence  to  yielding  a  negative- 
type  influence.  In  reviewing  the  means,  one  sees  that  a 
negative  weight  for  the  delay  tolerance  criterion  yields 
a  result  with  Group  3  having  the  higher  overall  value,  and  a 
negative  value  to  frequency  of  performance  gives  a  higher 
value  to  the  Group  1  tasks.  Thus,  the  effect  of  Function  1 
in  the  discrimination  analysis  of  the  factor  scores  is  to 
favorably  increase  the  discrimination  of  Group  3  through 
the  combination  of  the  consequences  and  immediacy  factors, 
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as  well  as  to  slightly  enhance  the  discrimination  of  Group  1 
tasks.  Again,  Figure  3  bears  this  out  in  placement  of  group 
centroids  along  the  x-axis  (Function  1).  Group  3  is  given 
a  high  positive  value,  while  Group  1,  though  slightly  nega¬ 
tive,  is  still  higher  than  Group  2  scores.  In  the  same 
analysis,  Function  2  credits  both  factors  (Immediacy  and 
Consequences)  with  positive  weights,  the  emphasis  being  on 
the  Immediacy  factor.  But,  this  function  is  of  little 
significance  whatsoever  in  the  discrimination  abilities  of 
either  function. 

Alternative  Measures 

The  selection  of  tasks  for  training  at  Radioman  Class 
A  School  involved  review  of  several  different  criteria 
which  were  carefully  rated.  However,  it  appears  as  though 
selection  may  be  done  just  as  well  from  the  use  of  only  a 
few  of  the  criteria  based  on  the  analyses  performed  for 
this  thesis.  The  factor-scored  discriminant  analysis  uses 
one  function,  involving  primarily  the  Consequences  factor, 
to  effect  a  classification  with  an  overall  accuracy  of 
61.6%.  The  percentage  of  correct  classification  of  both 
groups  of  tasks  which  are  taught  is  even  higher  than  the 
overall  classification  accuracy.  Two  selection  criteria 
are  dominant  in  the  consequences  factor;  namely,  probable 
consequence  of  inadequate  performance  and  time  between  job 
entry  and  task  performance.  Of  secondary  Importance  is 
frequency  of  performance.  In  a  recent  conversation  with  the 
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author,  Mr.  Richard  Lund  (1980)  of  Instructional  Program 
Development  Department  (IPDD),  San  Diego,  stated  that  although 
IPDD  did  not  perform  a  statistical  analysis  of  the  criteria 
for  the  selection  process,  the  most  influential  criteria  for 
task  selection  were  probable  consequences  of  inadequate 
performance,  the  time  criterion,  frequency,  and  decay  rate. 

It  is  interesting,  indeed  striking,  that  the  first  three 
criteria  mentioned  are  greatly  influential  in  the  Consequences 
factor.  It  seems  that  use  of  the  Consequences  factor  would 
increase  the  efficiency  of  the  criteria  rating  by  reducing 
the  number  of  independent  variables.  Prediction  can  be  made 
just  as  well  with  fewer  criteria  as  it  is  with  the  full  set 
of  criteria.  Thus,  a  small  number  of  criteria  has  the  ability 
to  do  all  the  work  of  several. 

Another  approach  which  merits  investigation  is  the  idea 
of  scaling  the  criteria  differently.  One  possibility  is  to 
create  a  weighted  checklist  of  statements  reflecting  the  full 
range  of  attitudes  toward  each  criterion.  Each  statement 
would  describe  one  attitude  toward  the  particular  criterion. 

A  panel  of  judges  would  classify  the  statements  in  categories 
ranging  from  those  they  consider  to  be  extremely  favorable 
to  those  they  consider  to  be  extremely  detrimental  (McCormick 
St  Tiffin,  1974).  This  classification  could  be  accomplished 
using  the  paired  comparison  technique  in  which  each  state¬ 
ment  is  compared  to  each  other  until  each  is  in  the  most 
appropriate  location  in  the  spectrum.  Weights  would  then  be 
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assigned  relative  to  the  final  positions  of  these  statements. 
Checklists  of  each  criterion  for  each  task  would  then  be 
distributed  to  the  SMEs.  Each  SME  could  indicate  which 
statements  most  accurately  depict  the  criterion  for  a  given 
task  by  placing  a  check  mark  next  to  the  appropriate  state¬ 
ment.  The  weights  of  the  statements  would  be  unknown  to 
the  SMEs.  Upon  receipt  of  the  completed  checklists  from 
each  SME,  the  selection  board  would  tally  the  weighted 
checks  and  determine  the  overall  rating  of  each  task  for 
a  training  decision.  This  type  of  Thurstone  scale  has  been 
widely  used  in  many  psychological  tests  and  for  applications 
relating  to  discrimination  of  judgment  (Bock  &  Jones,  1968). 
To  carry  this  technique  one  step  further,  the  SMEs,  instead 
of  simply  marking  the  appropriate  statements,  could  rank  the 
statements  using  the  paired  comparison  technique.  These 
rankings,  in  conjunction  with  the  previously  determined 
weights  could  further  refine  the  overall  ratings  of  the 
tasks  for  the  training  decision. 

Limitations 

This  analysis  focused  on  a  specific  application  of  the 
ISD  process.  For  a  broader  generalization  with  respect  to 
the  ISO  process  of  task  selection,  this  analysis  should  be 
cross- validated  with  other  schools,  e.g.,  other  Class  A 
Schools  and  advanced  Radioman  schools.  One  problem  that 
may  occur  is  that  different  sets  of  SMEs  may  alter  the 
reliability  of  the  results  resulting  in  shrinkage  of  the 
predictions. 
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This  analysis  concentrated  purely  on  task  selection. 

Use  of  the  JTI  and  application  of  the  ISD  process  appears  to 
be  fairly  good  in  determining  which  tasks  get  trained,  but 
this  does  not  mean  that  the  selection  is  optimum  for  the 
requirements  of  the  Navy.  Supervisors  and  managers  in  the 
operational  environment  perceive  training  in  relation  to 
the  end  product. 

In  the  task-selection  process  consideration  must  be 
made  of  how  the  graduate  is  able  to  function  in  his  assign¬ 
ment  after  graduation.  The  selection  process,  analyzed  by 
this  thesis,  utilized  the  results  of  a  NOTAP  survey  in 
determining  the  percentages  of  apprentices  performing  various 
tasks.  Further  follow-up  and  surveys  of  initial  assignments 
should  be  conducted.  These  surveys  should  be  performed  at 
the  sites  representative  of  and  in  proportion  to  the  types  of 
units  to  which  the  graduate  is  sent.  The  unique  needs  of 
the  fleet  should  be  noted  in  the  survey  for  use  as  feedback 
into  the  ISD  task  selection  process. 

Under  "ideal"  conditions,  40%  of  Radioman  Class  A 
School  graduates  report  to  shore  stations  while  the  re¬ 
maining  60%  report  for  sea  duty.  In  a  compilation  of  sea 
duty  assignments,  by  RM1  Randall  of  Radioman  Class  A  School, 
for  the  period  of  July  1979  through  December  1979,  the 
following  list  was  constructed: 
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Ship  Type 
Small  Combatants 


Percentage  Assigned 


30.96 

Amphibious  Ships  .  22.63 

Aircraft  Carriers  .  16.54 

Service/ Auxiliaries  .  15.52 

Large  Combatants . 8.52 

Submarines . 5.11 

Mine  Warfare . 73 


This  breakdown  in  assignments  could  be  used  for  determining 
which  tasks  deserve  increased  attention  (i.e.,  which  tasks 
are  more  applicable  to  a  higher  percentage  of  the  assign¬ 
ments)  as  well  as  for  future  survey  applications. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


The  results  determined  by  the  analytical  methods  of 
this  thesis  were  based  on  an  exceptionally  good  data  base. 

The  Data  Correlation  Sheets  used  were  the  same  ones  used  by 
the  actual  task  selection  board  for  the  Radioman  Class  A 
School  course.  The  ratings  assigned  to  each  criterion  for 
every  task  were  done  so  by  subject  matter  experts  intimately 
familiar  with  all  aspects  of  the  tasks.  These  subject 
matter  experts  were  actual  Navy  radiomen  representing  a 
broad  base  of  experience. 

Based  on  the  research  conducted  it  is  concluded  that : 

1.  Prior  to  this  study,  it  was  unclear  whether 
criteria  ratings  are  being  used  in  a  systematic  way.  It 
was  not  clear  whether  the  SMEs  were  able  to  purposely  rate 
certain  tasks  higher  to  increase  the  probability  of  selection 
for  training,  or  whether  the  actual  tasks  selected  for  training 
were  the  result  of  having  a  higher  rating  from  an  objective 
process.  This  thesis  has  demonstrated  that  criteria  ratings 
may  be  used  systematically. 

2.  It  appears  that  there  are  more  efficient/effective 
ways  of  evaluating  tasks  than  are  currently  being  used.  One 
possibility  suggested  is  the  development  and  application  of 

a  Thurstone  scale  to  criteria  rating.  This  type  of  scale 
yields  a  better  insight  as  to  the  exact  perception  the  SME 
has  of  the  tasks  than  the  current  scales  reveal.  Should 
this  scale  be  used,  more  efficiency  may  be  reaped  by  limiting 
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the  criteria  to  those  discovered  in  the  results  of  this 
research  which  carry  the  most  influence. 

3.  The  criteria  used,  in  and  of  themselves,  do  not 
reveal  a  substantial  discrimination  of  the  E-2  skill  level 
tasks  which  are  trained  from  the  E-2  skill  level  tasks  which 
are  not  trained.  There  are  apparently  other  factors  outside 
the  criteria  judged  which  play  a  part  in  making  this  deter¬ 
mination. 

4.  Results  of  this  thesis  are  not  necessarily 
generalizable  to  other  types  of  schools,  be  they  Class  A 
schools  or  other  Radioman-related  schools.  The  utilization 
of  the  criteria  ratings  in  the  same  fashion  may  not  be 
appropriate  for  other  schools,  nor  is  it  known  whether  the 
criteria  rating  scales  should  be  the  same  for  all. 

5.  There  may  be  simpler  ways  of  combining  criteria 
to  analyze  tasks.  For  the  specific  case  of  task  selection 
for  Radioman  Class  A  School,  the  several  individual  criteria 
can  be  consolidated  into  one  factor  which  works  about  as 
well  in  classification  of  tasks  which  should  be  trained. 

The  factor  concerned  is  the  Consequences  factor  which  is 
dominated  by  the  criteria  of  probable  consequence  of  in¬ 
adequate  performance  and  time  between  job  entry  and  task 
performance,  with  frequency  of  performance  acting  as  a 
secondary  source  of  influence.  With  its  adequate  prediction 
capability,  the  consequence  factor  may  be  used  in  task 
selection,  thus  saving  a  great  deal  of  time  and  resources. 
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Based  on  these  conclusions,  it  is  recommended  that: 

1.  Similar  procedures  as  employed  in  this  thesis 
should  be  used  to  evaluate  task,  selection  for  the  "classic" 

A  school  course.  The  results  of  such  analyses  may  prove 
quite  beneficial  in  terms  of  reducing  overall  ISD  costs. 

2.  Development  of  a  different  criterion  scaling 
procedure  that  would  be  simpler  and  more  effective  should  be 
considered.  The  Navy  might  be  better  off  in  its  training 
selection  decisions  if  a  Thurstone  scale  were  used  for 
measuring  the  tasks  in  lieu  of  the  current  rating  process. 
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APPENDIX  A 


Criteria  Ratings  of  Tasks  Analyzed 
from  the  Data  Correlation  Sheets 


Task 


Group*  Criteria* 


123456789 


Distribute  instructions  1 

Distribute  notices  1 

Distribute  directives  1 

Participate  in  field  days,  sweepdowns, 
and  similar  activities  1 


14  22225114 
14  22225114 
14  22225114 

81  33125115 


Participate  in  working  parties 

Participate  in  inspections 
(zone,  personnel,  safety) 

Participate  in  communications 

emergency  drills  (power  failure, 
casualty) 

Participate  in  communication 
exercises/ drills 

Participate  in  general  drills 


1  35  33225125 

1  36  33235124 

1  34  45344232 

1  42  45344232 

1  25  35335125 


Update  broadcast  files 

Update  general  message  files 

Update  world-wide  task 
organization  guide 


1  23  33225115 

1  21  22235215 

1  16  33235215 


Update  communications 
instruction  file 


1  27  22235114 


*A  key  for  the  group  entries  and  the  criteria  is  provided 
at  the  end  of  this  appendix. 
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Task 


Group 


Criteria 


Update  communications  notices 
file 

Update  communication  directives 
file 

Update  crypto  center  files 

Update  communication  center 
message  files 

Update  fax  files 

Update  message  tape  file 

Make  log  entries  in  general 
message  log 

Make  log  entries  in  the  broad¬ 
cast  checkoff  log 

Make  log  entries  in  the 
Morse  telegraphy  (CW)  log 

Make  entries  in  the  visual  log 

Make  entries  in  the  radio¬ 
telephone  log 

Make  entries  in  the  order-wire 
teletypewriter  log 

Make  entries  in  the  TTY 
ship/shore  log 

Receive  secret  material 

Control  secret  material 

Mark/ remark  secret  material 

Receive  confidential  material 

Control  confidential  material 


123456789 


1  24  22235114 

1  24  22235114 

1  24  33235115 

1  11  23235115 

1  24  33235115 

1  24  22225115 

2  50  22225115 

2  50  24225115 

1  50  44335224 

1  50  33324113 

1  50  44335235 

1  50  44335115 

2  54  44335115 

3  28  43234115 

3  28  43234115 

3  28  43234115 

3  28  43234115 

3  28  43234115 
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Task 


Group 


Criteria 


Mark/ remark  confidential  material 

Control  classified  working  papers 
(rough  drafts,  etc.) 

Control  classified  msc.  materials 
(ribbons,  etc.) 

Identify  &  correct  inadequate 
safety  precautions 

Cut  tapes  on  the  UGC-6  teletype 

Transmit  msgs  on  the  UGC-6  teletype 

Receive  msgs  on  the  UGC-6  teletype 

Perform  operator's  maintenance 
on  the  UGC-6  teletype 

Patch  with  the  SB-988/SRT 

transmitter  transfer  switchboard 

Perform  preventive  maintenance 
on  the  SB-988/SRT 

Receive  msg  traffic  on  the 

AN/FGC-100  TTY  terminal  equipment 

Perform  operator's  maintenance 
on  the  AN/FGC-100 

Pass  (.send)  message  traffic 
on  the  TT-333 

Perform  operator's  maintenance 
on  the  TT-333 

Perform  operator's  maintenance 
on  the  TSEC/KW  26 

Send  (transmit)  msg  on  the 
TT-333/UG  (dist.  trans  set) 


123456789 

3  28  42224115 

3  28  42224115 

3  28  43234124 

3  —  55334125 

2  56  24345235 

2  56  34225115 

2  56  34225215 

1  56  33225115 

2  06  22224115 

1  06  11115113 

2  04  34235215 

1  04  33235115 

2  04  24325115 

1  04  22225115 

1  08  43235125 

3  04  23225115 
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Task 


Group 


Criteria 


Monitor  the  AN/FGC-79A  (FLT  BCST 
(R))  FLT  SATCOM 

Receive  msg  on  the  AN/FGC-79  TTY 
(FLT  center  termination) 

Receive  msg  and  tape  on  the 
TT-331  A/UG  reperforator 

Set  up  the  November  System 
(multi-channel  receive) 

Secure  the  November  System 
(multi-channel  receive) 

Set  up  the  Charlie  System 
(UHF  covered  duplex) 

Secure  the  Charlie  System 
(UHF  covered  duplex) 

Set  up  Romeo  System  (.covered  voice) 

Secure  Romeo  System  (covered  voice) 

Set  up  Uniform  System  (.uncovered 
voice) 

Secure  Uniform  System  (.uncovered 
voice) 

Set  up  Golf  System  CHF  covered 
duplex) 

Secure  Golf  System  (HF  covered 
duplex) 

Select  channel  on  the  AN/SRA-12 
(elect  filter  assembly) 

Connect/disconnect  the  MF-HF 
antenna  (AS-25430)  to  the 
AN/SRA-12 
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123456789 

3  05  24115115 

3  05  24335115 

3  03  13115115 

3  17  43434244 

3  —  42434234 

3  17  42434244 

3  —  42334234 

3  17  42344234 

3  —  42334234 

3  17  32335234 

3  —  22235234 

3  17  42434234 

3  —  42334234 

2  14  22335115 

1  34  22225115 
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Task 


Group 


Criteria 


123456789 


Connect/disconnect  the  MF-HF 
receiver  (R1051)  to  the 

AN/SRA-12  2  33  22225115 

Patch/unpatch  the  SB  973  receiver 

transfer  switchboard  2  15  12245225 

Patch/unpatch  the  SB-1203A/UG 

communication  patching  panel  2  18  32235125 

Perform  preventive  maintenance 

on  the  SB-1203A/UG  1  18  32235113 

Patch/unpatch  the  SB-1210A/UGO 

communication  patching  panel  2  20  32235125 

Perform  preventive  maintenance 

on  the  SB-1210A/UGO  1  20  32235113 

Energize  the  PP-3495/UG 

power  supply  1  —  12115114 

Deenergize  the  PP-3495/UG 

power  supply  1  —  12115114 

Patch/unpatch  the  SB-988/SRT 

transmitter  transfer  switchboard  2  06  12235125 

Patch/unpatch  the  SB-863  trans¬ 
mitter/transfer  switchboard  2  14  12235125 

Adjust  the  AN/URA-17 

(comparator-converter  group)  2  24  22235225 

Tune  the  AN/SRA-13  (coupler  group 

antenna)  1  04  43445235 

Energize  the  AN/UCC-l  telegraph 

terminal  2  13  11215114 

Deenergize  the  AN/UCC-l  telegraph 

terminal  2  1311215114 
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Task 


Group 


Criteria 


Start  the  CV-2460/SGC  telegraph- 
telephone  signal  converter 

Set  up  the  CV-2460/SGC  telegraph- 
telephone  signal  converter 

Receive  messages  on  the  AN/UGC-25 
teletype 

Operate  the  AN/SRA  33  (.antenna 
coupler  group) 

Adjust  volume  on  the  AM-3729/SR 
(speaker  amplifier) 

Patch/unpatch  the  C-7594  (remote 
switching  control) 

Set  up  the  AN/URT-23(V)  radio 
transmitter  set 

Tune  the  AN/URT-23(V)  radio 
transmitter  set 

Check  and  adjust  the  TSEC/KWR-37 
(crypto  unit) 

Perform  start  sequence  on  the 
TSEC/KWR-37 

Establish  cipher  communications 
on  the  TSEC/KW-7  (crypto  unit) 

Record  message  transmission  in  the 
outgoing  circuit  log 

Record  time  of  delivery  on  the 
transmitted  message 

Record  message  reception  in  the 
incoming  circuit  log 

Record  time  of  receipt  on  the 
received  message 


123456789 

2  00  11215114 

1  00  22335235 

2  25  32225115 

2  04  22335225 

2  02  12215115 

3  00  22222125 

2  12  32335235 

2  12  32325235 

3  26  42344235 

3  26  42344235 

2  46  42335235 

2  50  24225115 

2  28  23235115 

2  50  24235115 

2  28  23235115 
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Task 


Group 


Criteria 


123456789 

Record  message  reception  on  the 

broadcast  checkoff  sheet  2  50  22235115 

Perform  sequence  of  operation  on 

the  R1051  radio  receiver  2  33  23335235 

Perform  sequence  of  operation  on 

the  SSR-1  (VHF  multi-channel)  3  —  43222124 

Sequence  outgoing  original  msg  draft  2  —  44225115 

Assign  date/time  group  and  SSN  to 

outgoing  message  draft  2  45  15225115 

Screen  and  review  original  outgoing 

message  draft  2  38  24445345 

Format  outgoing  msg  for  entry  into 

TTY  tape  emission  media — Autodin  1  —  14445345 

Format  outgoing  msg  for  entry  into 
TTY  tape  emission  media — Modified 

126  2—14445345 

Format  outgoing  msg  for  entry  into 

TTY  tape  emission  media— ACP  126  2  —  14444534 

Format  outgoing  msg  for  entry  into 

voice  transmission  systems  1  —  24445334 

Format  outgoing  msg  for  delivery 

by  CW  transmission  media  1  —  44445335 

Format  outgoing  msg  for  delivery 

by  scanner  (LDMX)  system  2  —  14545335 

Type  smooth  formated  message  draft  2  52  24325135 

Screen  &  review  smooth  copy  of  out¬ 
going  messages  for  completeness 

ft  correctness  3  20  24322225 

Deliver  outgoing  smooth  msg  to 

releasing  officer  for  release  1  26  44525115 
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Task 


Group 


Criteria 


Deliver  outgoing  message  to 

selected  circuit  for  transmission 

Screen  &  review  outgoing  trans¬ 
mitted  msg  from  circuit  operator 

Log  outgoing  message  information 
into  central  msg  log 

Route  outgoing  msg  into  action, 
info  departments 

Reproduce  copies  of  outgoing  msg 
IAW  route  stamp 

Stamp  outgoing  message  with 
designated  classification 

Distribute  reproduced  copies 
of  orig  msg 

Check  outgoing  msg  prior  to  filing 
(traffic  checker) 

File  outgoing  messages  into  comm 
cen  files 

Advance  route  designated  incoming 
msgs 

Screen  &  review  incoming  messages 

Log  incoming  msgs  into  central 
message  log 

Reproduce  incoming  messages 

Distribute  incoming  messages 

Check  incoming  messages  prior  to 
filing  (.traffic  checker) 

File  incoming  msgs  into  com 
cen  files 


123456789 

2  —  44525115 

3  20  24322225 

2  54  12225115 

2  26  22225115 

1  60  22225115 

2  56  44215115 

1  40  13225115 

3  24  43332125 

2  11  33255115 

2  —  45225115 

2  44  45325135 

3  54  13235115 

1  60  13225115 

1  40  13225115 

2  24  33255115 

3  11  43332125 
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Task  Group  Criteria 


123456789 


Accept  message  typed  on  Form 
DD-173  for  input  to  LDMX/ 

NAVCOMPARS  3  —  15515115 

Reject  message  on  Form  DD-173 

(OCR  operator)  3  —  13315125 

File  service  messages 

(service  area  coordinator)  3  11  11115115 


KEY 

Groups 

1.  Tasks  at  the  E-2  skill  level  which  were  not  taught. 

2.  Tasks  at  the  E-2  skill  level  which  were  taught. 

3.  Tasks  above  the  E-2  skill  level  which  were  taught. 

Criteria 

1.  Percentage  of  E-2s  to  E-4s  Performing  Task. 

2.  Probable  Consequence  of  Inadequate  Performance. 

3.  Delay  Tolerance. 

4.  Learning  Difficulty. 

5.  Probability  of  Deficient  Performance. 

6.  Time  Between  Job  Entry  and  Task  Performance. 

7.  Decay  Rate. 

8.  Complexity . 

9.  Frequency  of  Performance. 
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